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Introduction 



Early in 2003, the National Assessment Governing Board established the National Commission on 
NAEP 12 th Grade Assessment and Reporting, following the recommendation of retiring Governing 
Board Executive Director Roy Truby. The Governing Board’s charge to the Commission was 

“To review the current purpose, strengths, and weaknesses of 12 th grade assessment and 
reporting by the National Assessment of Educational Progress (NAEP) and set forth 
recommendations for improvement to the National Assessment Governing Board.” 

The Commission was instructed to focus on improvements needed for 12 th grade NAEP and not on 
the status of high schools generally or the quality and rigor of the high school curriculum. The 
Commission has attempted to heed this guidance scrupulously. 

Commission members were appointed by Darvin Winick, Chairman of the Governing Board. The 
eighteen Commission members were drawn from K-12 and higher education faculty and 
policymakers, the business community, and the military. Thus, the composition of the Commission 
includes both those who “produce” high school graduates and those who receive them (see 
Appendix A). The members have varying areas of expertise and a wide range of views, but share a 
common interest in the quality of the students produced by our nation’s education systems. 

Commission members’ knowledge of NAEP prior to appointment ranged from very much to very 
little. However, over the course of the meetings, Commission members’ familiarity with NAEP has 
increased and their appreciation of the singular role that NAEP plays in American education has 
deepened. The members of the Commission recognize NAEP’s quality, integrity, and unique role 
as an independent, external measure of student achievement at the national and state levels (see 
Appendix B). 

The Commission has met and deliberated five times since March 2003. It focused on answering the 
following question — What do we know about 12 th grade achievement, what do we need to know, 
and what can NAEP provide? The Commission received expert testimony from twenty-four 
witnesses, commissioned and reviewed six white papers and two research studies, conducted a 
focus group with the Council of Chief State School Officers, and received comments on preliminary 
Commission recommendations from the National Association of State Boards of Education and 
Tennessee state education leaders (see Appendix C). 

Time for Bold Action 



The Commission members have concluded that 12 th grade NAEP should be redesigned. Twelfth 
grade NAEP has the potential to supply crucial information about student achievement that America 
needs and that is unavailable from any other source, but NAEP is not now fulfilling that potential. 
NAEP’s leaders have an opportunity to invigorate 12 th grade NAEP by making changes that would 
increase its relevance and usefulness to policymakers and the public. 

The testimony of expert witnesses and the comments of state leaders in education and the business 
community were consistent, forceful, and persuasive. They told the Commission that (1) it is 
important to continue to report 12 th grade NAEP results; (2) it is essential to expand state NAEP to 
include 12 th grade, not just grades 4 and 8 as is the case now; and (3) it would be invaluable for 
NAEP to report on the readiness of 12 th graders for college, training for employment, and entrance 
into the military. 
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State education leaders told the Commission that they want to know how well their 12 th graders 
measure up academically and how ready they are for higher education and work world challenges 
they will face after high school. They want to compare the achievement of their 12 th grade students 
to those in other states and have a national benchmark to compare against as well. State education 
leaders described this information about 12 th graders as the “missing piece” that they presently do 
not have, but need, and said that only NAEP has the ability to provide this information. 

State education leaders were insistent about the importance of this information because they know 
what is at stake. They reported to the Commission that about 45-55 percent of college freshmen 
entering 4-year and 2- year institutions are under-prepared for college-credit coursework and thus 
must be enrolled in non-credit, remedial courses in reading, writing, and mathematics. Students 
who are initially enrolled in remedial courses are the least likely to graduate from college and most 
likely to suffer the consequences of lessened employment opportunities and income in comparison 
to their peers with college degrees. 

Armed with state-comparable information about the readiness of 12 th graders for postsecondary 
pursuits, K-12 and postsecondary policymakers believe they will be better able to identify and 
address education problems. State policymakers also know that the quality of the education system 
and the readiness of the workforce are key factors business leaders consider. Having good 
information about 12 th grade student achievement not only can help in making better education 
policy decisions, but can have an impact on the states’ long-range employment/economic outlook as 
well. 

Summary: A New Vision for 12 th Grade NAEP 

A bold, new vision for 12 th grade NAEP is needed. The Commission members’ recommendations 
for 12 th grade NAEP follow below. The changes proposed are substantive and significant. Some 
may require new legislation. Expanding state NAEP from grades 4 and 8 only to grades 4, 8, and 
12 will require additional appropriations. It also will require dramatically new approaches for 
increasing school and student participation in NAEP at the high school level and enhancing the 
motivation of students to do their best in taking NAEP. Providing information about the readiness 
of 12 th graders for college, training for employment, and entrance into the military will require 
revisions to NAEP test frameworks, test questions, achievement levels, and reporting procedures. 
Making these revisions, too, may require additional appropriations and will take time and hard work 
to achieve. However, the Commission members believe that the modest investment needed would 
be far out- weighed by the enormous benefits. 

It is crucial that NAEP’s leaders change 12 th grade NAEP to: 

• provide 12 th grade state level results, and 

• report on readiness for college, training for employment, and entrance into the military 

America needs to know how well prepared its high school seniors are to become productive citizens 
and to compete in a global economy — how well they can read, write and compute, and what they 
know about science, history, civics, and other important disciplines. America’s state and national 
leaders, and indeed the American public, need this information to plan now for education in the next 
decade and beyond. Only the National Assessment of Educational Progress can provide this 
information — for the nation and for states — and it is necessary for our nation’s continued well being 
that it be provided. 
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Recommendations 



1. The National Assessment of Educational Progress (NAEP) should provide each state 
with results on the achievement of its 12 th graders. All states should participate in 
reading and mathematics assessments in grade 12 as they do now in grades 4 and 8. 

The Commission members believe that expanding state-NAEP to include grade 12 and requiring all 
states to participate in reading and mathematics at grade 12 are the most important changes that can 
be made to 12 th grade NAEP, and where the value added, service to the public, and opportunity to 
benefit students, education institutions and the business community will be greatest. 

NAEP 12 th Grade Student Achievement Results Are Important 

The 12 th grade 1 NAEP student achievement results are important and should continue to be 
provided to the public. They are important because they represent a measure of achievement at the 
endpoint of compulsory education and the transition to responsible occupations and civic 
responsibilities. Through their high school experience, young adolescents develop into young 
adults, destined for adult pursuits — college, postsecondary training for employment, the military 
and, ultimately, the management of our nation’s affairs. What do 12 th graders know? What can they 
do? Is this year’s group of 12 th graders better or less well prepared than previous groups of 12 th 
graders? It is essential that the public have answers to these questions. Only NAEP is positioned to 
provide such answers — for the nation and, potentially, for individual states. 

The Commission members considered other options for defining “who is tested” for NAEP 
assessments at the high school level, including: 10 th graders, 11 th graders, 15-year-olds, and 17- 
year-olds, to name a few. These alternatives lack the relevance and usefulness of grade 12 because 
it is the endpoint of the K-12 education system and a key transition point to adulthood. 

NAEP is the only source of nationally representative data on student achievement at 12 th grade. 
States typically do not test 12 th graders and even if they did, state tests, with their varying content 
and administration procedures, could not be “added up” to get a national result. College admission 
tests, such as the ACT and SAT, are taken by a self-selected subgroup of college bound high school 
juniors and seniors and thus do not yield nationally representative results for 12 th grade students. 
There are no available means for measuring the results of K-12 education in the United States other 
than by using NAEP. 

NAEP 12 th Grade State Results Are Needed 



National 12 th grade student achievement results are important because 12 th grade is the endpoint of 
compulsory education and the transition point to college, postsecondary training for work, and the 
military, but state 12 th grade NAEP results may be even more important. 



1 The term “12 th grader” as used in NAEP means a student who has completed and is currently taking sufficient course 
work to be eligible to graduate from high school at the end of the current school year. Whether a student meets this 
criterion is determined by the school according to state and local policies; there is no NAEP definition of what 
combination of courses is adequate for designation as a “12 th grader.” The Commission members believe that leaving 
the determination to local school authorities, interpreting local policy, is the correct approach. 



3 




Education governance and policymaking are primarily the province of state and local authorities. It 
is the states’ responsibility to provide elementary/secondary education, higher education, and 
postsecondary training. Thus, the views of state officials about 12 th grade state NAEP must be 
given great weight. The Commission members heard from a sample of chief state school officers, 
state board of education members, representatives of governors, and other state education and 
business leaders who expressed their reasons for supporting 12 th grade state NAEP. 

State education leaders said that they want a periodic, external measure of 12 th grade student 
achievement. They want to compare 12 th grade achievement results and performance standards of 
their state — including high school graduation standards — with those of other states and the nation 
and use the results to help make informed decisions about state-based high school improvement and 
reform initiatives. Furthermore, state education leaders support NAEP in this role because they 
recognize NAEP’s unique capability to serve as a trustworthy, stable, independent means of 
monitoring 12 th grade progress over time. 

Only NAEP provides both a consistent measure of student achievement across states and a national 
reference point for comparison. The dozens of different state-developed tests and standards are 
designed to measure within-state results, but are unsuitable for cross-state and national 
comparisons. NAEP gives states — and the public — important information about their standards and 
assessments in comparison to those of other states and to the nation overall. State tests are 
developed separately, have differing content, are administered according to varying procedures, 
apply different standards, and report at different times. State tests, therefore, cannot be compared to 
each other. However, because NAEP uses common test questions, testing procedures, and reporting 
measures across states and for the nation, each state that wishes to do so can use NAEP to compare 
its results with those of other states and to the nation overall. 

This e pluribus unum aspect of NAEP is its great strength. This strength is especially well realized 
at grades 4 and 8 in reading and mathematics, where participation of all 50 states is required under 
the No Child Left Behind Act. This strength is not realized at all at grade 12, because NAEP does 
not provide state results at grade 12. In fact, no state has a measure that represents the achievement 
of all its 12 th graders. While many states have test requirements that must be passed to graduate 
from high school, they are usually taken by the end of 10 th or 1 1 th grade, with multiple chances for 
passing by 12 th grade. They are not a measure of 12 th grade achievement. Only NAEP is positioned 
to provide comparable state results at the 12 th grade. If made available, state-NAEP would permit 
states to see how well their 12 th graders meet NAEP’s “Basic” and “Proficient” performance 
standards and how their 12 th graders compare with peers in neighboring states in subjects such as 
reading, mathematics, science, and writing. 

Participation in NAEP 12 th Grade State Assessments in Reading and Mathematics Should be 
Mandatory and in Science and Writing Should be Voluntary 

The Commission members believe that 12 th grade state NAEP achievement results would provide a 
needed public service that, based on testimony and comments received, state policymakers will find 
attractive and desirable. State-level participation at grade 12 should be mandatory for schools in 
reading and mathematics and voluntary in science and writing, as it is at grades 4 and 8. And, as it 
is at grades 4 and 8, the cost of participation in state NAEP at grade 12 should be borne by the 
federal government. 
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Although voluntary participation in state NAEP at grade 12 in any subject is already authorized, 
making participation in state NAEP mandatory at grade 12 in reading and mathematics will require 
legislative authorization. 

The state-NAEP program is conducted at present only at grades 4 and 8. Assessments in reading 
and mathematics at these grades occur every two years, with all states required to participate under 
the provisions of the No Child Left Behind Act and the costs of NAEP testing and reporting borne 
by the federal government. As with the 4 th and 8 th grade, state-level NAEP at the 12 th grade would 
provide a common measure for states that want to compare results across their disparate state 
assessment systems. The logic of the No Child Left Behind Act suggests that the quality of 
achievement at the end of the K-12 education system, not just progress along the way, should be the 
ultimate focal point for measurement. But more than legislative logic, participation of schools in 
NAEP 12 th grade state-level assessments in reading and mathematics should be mandatory because 
of the value and importance of having 12 th grade student achievement results. 

Having good information about what 12 th graders know and can do is important for what it reveals 
about the results of K-12 schooling and the readiness of emerging generations to assume capably 
the reins of adult responsibilities. Such information can be used by state policymakers to help make 
informed decisions about matters as diverse as: K-12 education standards; the respective need for 
“developmental” and college-credit instructional classes in higher education; the likely number of 
individuals qualified for high-skill technical training for employment; the degree to which core 
knowledge about democratic institutions and principles has been absorbed; and the quality of the 
pool of potential military recruits. 

Mandatory participation in 12 th grade state NAEP will have a beneficial side effect: improving the 
participation rate of high schools in NAEP. The low participation rate at the 12 th grade is a problem 
that must be addressed (see discussion below in Recommendation 3). The requirement to 
participate in state NAEP in reading and mathematics at grades 4 and 8 was first implemented in 
2003 and had positive effects on participation. With mandatory participation of schools, the overall 
participation rate of students increased from 80% to 93% at grade 4 and from 78% to 94% at grade 
8 between the 2002 and 2003 assessments. It is very likely that mandatory participation of schools 
in reading and mathematics assessments at grade 12, together with a system of incentives (also 
discussed in Recommendation 3, below), would result in a satisfactory 12 th grade NAEP 
participation rate. 

Subjects other than reading, mathematics, science, and writing should be assessed at the 
national level only, if resources permit. 

Achievement in reading, mathematics, science, and writing are important overall indicators of 
academic readiness for pursuits beyond high school. Within the available resources for 12 th grade 
NAEP, these subjects should be assessed first. However, these subjects alone do not encompass the 
knowledge and skills necessary for contemporary life nor do they reflect the range of the high 
school curriculum. Subjects such as history, geography, civics, foreign language, the arts, and 
economics also are included in the NAEP schedule of assessments; the Commission suggests this 
ordering of subjects for consideration as a priority ranking. The Commission members recommend 
that NAEP should continue to conduct assessments at grade 12, at the national level only, in 
subjects other than reading, mathematics, science, and writing, according to priorities and a 
schedule determined by the National Assessment Governing Board. 
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2. NAEP should report 12 th grade students’ readiness for college-credit coursework, 
training for employment, and entrance into the military. 

The Commission members believe that the second most important new direction for 12 th grade 
NAEP is to report on readiness for postsecondary learning. Since 1969, NAEP has reported what 
students know and can do in a range of subjects commonly taught in elementary and secondary 
schools. NAEP should continue to meet this purpose in reporting 12 th grade results. However, 12 th 
grade is the major point of transition to higher education, training for employment, and entrance 
into the military. There currently is no dependable source of information about the readiness of 12 th 
graders for these destinations, but that information is critically important and NAEP is uniquely 
positioned to provide it. 

Thirty-five years ago and more, a high school diploma was the primary target of education. At the 
time, it was considered a sufficient prerequisite for getting a job that could sustain an individual and 
a family and adequate preparation for meeting civic obligations. Today, this is hardly true at all. 

The earnings gap between those with only a high school diploma and those with postsecondary 
education and training has widened substantially and the technological/scientific, legal, and moral 
complexity of today’s public policy issues require more to be an “informed citizen” in a democracy. 

As our economy moves increasingly from production to service, from manufactured goods to 
information, from line workers acting in isolation following specific instructions to workers acting 
collaboratively making independent decisions, and as using technology becomes a part of more job 
descriptions, a high school education will no longer suffice. 

Similarly, expectations are changing for the majority of students about the endpoint of formal 
education. Today, approximately 88 percent of 8 th grade students say that they want to attend higher 
education and about 70 percent of high school graduates do enroll in postsecondary education and 
training within two years after graduating. “ However, possession of a high school diploma in itself 
often does not equate with the competencies needed to be ready for college credit course work or 
for training for high performance jobs. Many students are finding themselves accepted into college, 
but — often to the surprise of their parents and themselves — enrolled in “remedial” or 
“developmental” classes in reading, writing, and mathematics. This adds costs to a college 
education, increases the time needed to earn a degree, and has the greatest negative impact on low- 
income students — the very students most at-risk of not finishing. NAEP would provide an 
important and needed service by reporting on the readiness of 12 th graders for postsecondary 
learning — that is, for higher education, training for employment, and entrance into the military — 
and by illustrating in its reports the competencies that reflect such readiness. 

Reporting readiness for pursuits beyond high school is an important new direction for NAEP and a 
complex undertaking that will require considerable research to implement and validate. For 
example, the definition of readiness for college varies significantly among institutions across the 
country. Likewise, the definition of readiness for training for employment will vary, given the vast 
array of jobs that exist and the respective skills, abilities, and knowledge required to perform them. 
Carrying out a process to arrive at definitions of “readiness” will be lengthy, and simple, “one size 
fits all” definitions may not be possible to achieve. 



2 Kirst, Michael; “College Preparation and Grade 12 NAEP”; paper prepared for the National Commission on NAEP 
12 th Grade Assessment and Reporting; August 2003. 
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Reporting on “readiness” involves the introduction into NAEP of predictive statements about 
student performance. Using predictive statements would move NAEP reporting from the current 
approach of looking “backwards,” describing what 12 th grade students know and can do as 
measured by NAEP, to “forecasting” what particular levels of NAEP achievement imply about the 
adequacy of preparation for various postsecondary pursuits (i.e., college, training for employment, 
and entrance into the military). 

To include predictive statements about post high school readiness in NAEP reports, appropriate 
validation research would have to be conducted. This would involve studies of college students, 
individuals in job training of various kinds, military recruits, and cohorts of students who took 
NAEP, using a longitudinal research design. Clearly, such studies would be costly, take significant 
time to execute, and may require congressional authorization. 

In addition, it is likely that NAEP test frameworks, specifications, test questions, and achievement 
levels would need to be revised. 

NAEP assessment frameworks describe the content to be tested in each subject area assessment and 
the types of questions to be asked, whether multiple choice, essay, or short answer. The 
specifications are more detailed instructions used by the test developers in constructing individual 
test questions and the assessment as a whole. The result is a set of test questions used on the 
assessment. 

The NAEP achievement levels are standards used for reporting NAEP results that describe 
performance at the “Basic,” “Proficient,” and “Advanced” levels as determined by the National 
Assessment Governing Board. Each achievement level consists of a range on the NAEP scale, a 
description of what students in that range know and can do, and illustrative test questions. 

The current NAEP 12 th grade assessments are focused on content generally taught by 12 th grade and 
are not specifically designed to support inferences about the readiness of 12 th graders for higher 
education, for training for employment, and for entrance into the military. While a preliminary 
expert review of NAEP 12 th grade assessments in reading, writing, and mathematics indicated that 
performance in reading at the “Proficient” level may indicate readiness for college credit course 
work, this was not found to be true for writing and mathematics and would require further research 
to validate. The expert review noted that some skills deemed important in determining readiness 
for higher education are not included in NAEP (e.g., editing skills in writing). Expert reviews of 
NAEP’s current capacity to report on readiness for work drew similar conclusions. 4 The NAEP 
frameworks, specifications, test questions, and achievement levels should be reviewed with the 
objective of supporting such inferences and revised accordingly. 



3 The Education Trust; "What Can Grade 12 NAEP Tell Us about Readiness for College-level Work?”; study conducted 
for the National Commission on NAEP 12 lh Grade Assessment and Reporting; September 2003. 

4 Carnevale, Anthony P. and Desrochers, Donna M.; “Considerations in Using 12 th Grade NAEP as a Prospective 
Indicator of Readiness for College and Employment”; paper prepared for the National Commission on NAEP 12 lh 
Grade Assessment and Reporting; October 2003. 

ACT, Inc; “Analysis of Commonalities and Gaps: Comparing the National Assessment of Educational Progress to ACT 
WorkKeys Assessments”; study conducted for the National Commission on NAEP 12 th Grade Assessment and 
Reporting; September 2003. 
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The Commission members believe that reporting on “readiness” will make NAEP more useful to 
the public and policymakers at the K-12 and postsecondary levels. However, the accomplishment 
of this new purpose for NAEP faces many foreseen and unforeseen hurdles. The Commission 
members believe that the move towards reporting on “readiness” should be viewed as a process of 
change in NAEP, to evolve over time, implement with care, and validate with diligence. 



3. NAEP’s leaders — the National Assessment Governing Board, the U.S. Department 
of Education, and the National Center for Educational Statistics — should develop and 
implement bold and dramatically new incentives to increase the participation of high 
schools and 12 th grade students in NAEP and the motivation of 12 th grade students to 
do their best on NAEP. 

The National Assessment is widely respected as a trustworthy indicator of student achievement at 
grades 4, 8, and 12, providing the American public data of exceptional quality. However, this 
respect is at serious risk for the NAEP 12 th grade information. The risk arises from the low 
combined participation rate of schools and students at the 12 th grade in the NAEP sample. The low 
combined participation rate is a significant threat to the validity of NAEP 12 th grade student 
achievement results. The lower the NAEP participation rate is, the less precise the estimates of 
student achievement produced by NAEP. With about one-third to one-half the students selected for 
the 12 th grade sample not participating, the Commission members view the combined school and 
student rates at the 12 th grade as too low. 

High schools selected for the NAEP sample consistently participate at lower rates than elementary 
and middle schools. For example, the school participation rates in 2000 were 89%, 85%, and 82% 
for 4 th , 8 th and 12 th grades, respectively. Likewise, many fewer 12 th graders drawn for the sample 
within the schools that do participate actually take NAEP, in comparison to 4 th or 8 th graders. In 
2000, student participation rates were 96%, 92%, and 77% for 4 th , 8 th , and 12 th grade, respectively. 

While the separate school and student participation rates appear respectable, the combined 
school/student participation rates for 12 th grade are not. Over the period 1988 — 2000, the combined 
school and student participation rates at the 12 th grade have hovered around 65%, meaning that 
about one-third of the student sample did not participate. However, in 2002, the 12 th grade rate 
dropped precipitously to 55%, meaning that almost one-half of the students drawn for the 
original sample did not participate (Table 1). The Commission members believe that action must 
be taken to improve the school and student participation rates at the 12 th grade. 

Table 1 

Combined School and Student Participation Rates in NAEP by Grade and Year 

1988—2003 





1988 


1990 


1992 


1994 


1996 


1998 


2000 


2002 


2003 


Grade 4 


82% 


82% 


81% 


80% 


82% 


84% 


85% 


80% 


93% 


Grade 8 


76% 


77% 


76% 


78% 


76% 


81% 


78% 


76% 


94% 


Grade 12 


65% 


66% 


66% 


64% 


64% 


65% 


64% 


55% 


* 



* Only grades 4 and 8 were tested in 2003. All states were required to participate in grades 4 and 8 reading and mathematics under 
the No Child Left Behind Act. 
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The Commission members acknowledge that, by definition, “12 th grader” includes only in-school 
youth, and not the significant numbers who have dropped out of high school by 12 th grade. The 
dropouts, together with NAEP’s low participation rates, result in only between 45% and 54% of the 
“potential” sample actually being represented in the NAEP 12 th grade sample. Reducing dropouts is 
the responsibility of state and local officials, not NAEP. Nonetheless, it is essential that NAEP’s 
leaders take significant steps to improve school and student participation rates for grade 12 NAEP. 

Steps to improve participation may include intense public information campaigns, special 
recognition certificates for schools and students that participate in 12 th grade NAEP, redeemable 
certificates for students and schools that participate, college scholarships, acknowledgement of 
NAEP participation on student transcripts, and other tangible incentives for participation. The 
NAEP “participation strategy” should be informed by research and focus group studies on the 
effectiveness of various incentives, conducted with 12 th grade students who have participated in 
NAEP and with high school principals, beginning with the 2004 NAEP assessments. (An 
illustration of a participation strategy is provided at Appendix D.) 

The common wisdom is that 12 th graders, being “test wise,” suffering from “senioritis,” and having 
received college acceptances by the spring of their senior year, do not take “no-stakes NAEP” 
seriously when asked to participate in February or March of their senior year, the time scheduled for 
NAEP testing. This common wisdom suggests that NAEP may be underestimating 12 th grade 
achievement due to low student motivation. It also has been noted that 12 th graders leave more 
blank answers for essay type questions than for multiple choice type questions, another potential 
source of underestimation of achievement that may be related to student motivation. 

Although there is anecdotal evidence that 12 th graders may not be performing their best on NAEP, 
there is no conclusive scientific evidence that this is true. And if it is true, there are no data to 
provide clues about the factors that may be depressing performance or how those factors might be 
overcome. However, a recent study by the National Research Council 5 finds that student 
“disengagement” with respect to academics increases as students increase in grade, is pervasive in 
high schools, and is affected by many school-controlled factors. Although the Commission 
members do not believe that NAEP can or should take on the overall task of addressing student 
“disengagement” in high schools generally, it is a factor that NAEP’s leaders must take into account 
in addressing school and student participation. 

Research should be conducted to determine the motivation of 12 th graders taking NAEP and on 
ways to enhance participation in NAEP of both students and schools. Examples of possible topics 
include: developing observable indicators of student engagement in taking NAEP and measuring 
student engagement against those indicators; identifying impediments to school participation 
(e.g., sampling procedures and time of year of testing) and ways to reduce those impediments; 
evaluating the effectiveness of different incentives for participation; and determining whether low 
completion rates on questions where students must respond by writing sentences or a paragraph 
signal low student motivation. 



5 “Engaging Schools: Fostering High School Students’ Engagement and Motivation to Learn”; National Research 
Council, 2003. 
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Sufficient resources, imagination, and hard work will be needed to achieve significant 
improvements in 12 th grade school and student participation in NAEP. Half-hearted or lackluster 
efforts will not be adequate, nor will efforts that fail to engage all who are affected — students, 
teachers, principals, administrators, and many others — in a coordinated approach. Mandating 
participation in 12 th grade state NAEP, as suggested in Recommendation 1, above, will do much to 
improve school participation and, over time, may help somewhat with student participation and 
motivation, but more will be needed. 

Although high school student “disengagement'’ and “senioritis” may be real, these are maladies that 
others are properly positioned to address and should not be used as an excuse for non-action where 
NAEP is concerned. Appeals can be designed that will be attractive to students and principals and 
that have power to increase school and student participation. Without effective intervention, 12 th 
grade participation rates are likely to get worse. Ever-lower participation rates will compromise 
NAEP’s quality and credibility. Therefore, NAEP’s leaders must focus seriously on increasing 
school and student participation in 12 th grade NAEP. 



4. NAEP 12 th grade student achievement results should be given prominence in NAEP 
reports. 

The Commission members believe that NAEP 12 th grade student achievement results are especially 
important and should be presented in ways that capture public attention. Whether through separate 
chapters in NAEP reports, separate reports limited to 12 th grade, and other effective means, NAEP 
12 th grade data deserve to be made more prominent. Currently, NAEP 12 th grade results are 
reported along with 4 th and 8 th grade results. Tables, charts, and graphs in NAEP reports usually 
display results for grades 4, 8, and 12 together. Report text treats grades 4, 8, and 12 alike. This 
form of reporting has proven useful and is not recommended for elimination. However, the 
Commission members believe that the importance of 12 th grade student achievement demands new 
reporting policies and procedures that will highlight 12 th grade results. 

The achievement of 4 th graders and 8 th graders is important, as it represents progress towards the 
endpoint of compulsory education for most students — 12 th grade. However, it is the 12 th grade that 
is the transition point to higher education, training, employment, and the military. As such, the 
achievement of 12 th graders is an indicator of the nation’s potential “human capital” captured at a 
decisive moment. How many 12 th graders are ready for a challenging higher education, training for 
high performance/high pay jobs, demanding assignments in the military, or only low-skill 
employment? Skill in reading, mathematics, science, and writing are among the primary proxies 
used for assessing general readiness for entrance or acceptance by institutions of higher education, 
training organizations, employers, and the military. Therefore, the Commission members believe 
that NAEP would provide an important service to policymakers and the public by providing reports 
that highlight the achievement of 12 th graders, especially in subjects such as reading, mathematics, 
science, and writing. 
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5. NAEP 12 th grade transcript studies should be continued, the types of information 
collected should be expanded, and the dissemination of results should be improved. 

Since 1987, NAEP has periodically collected data from high school transcripts of students 
participating in the 12 th grade assessments. While these transcript studies hold the potential for 
providing powerful insights into the rigor of the high school experience, the promise of these data 
collections has never been realized. Policy oversight and guidance of the transcript studies should 
be heightened and strategies for dissemination should be improved so that the most important, 
policy relevant data are collected and analyzed and the results receive appropriate public attention. 

The transcript data include courses taken and grades earned. From these data, changes in course- 
taking patterns over time can be monitored. For example, the NAEP transcript data indicate that 
between 1982 and 1998, the mean number of course credits taken increased, from 21.6 to 25.1, with 
most of the increase being in academic courses. Over this period, course taking increased in 
mathematics (.9 credits), English (.2 credits), science (.9 credits), social studies (.5 credits), and 
foreign language (.9 credits). These are important findings, suggesting that increasing numbers of 
high school students are taking more courses and, possibly, more rigorous coursework. 

There is great potential in the NAEP transcript data. Many useful analyses could be conducted, 
including: the relationships between courses taken and student achievement — both in terms of grade 
point average (GPA) and performance on NAEP; the content and/or academic rigor of the 
coursework being taken; the relationships between high school exit exams, the number of times 
taken, and achievement; information on programs of joint enrollment in high school and college 
courses; the quality or rigor of coursework taken during the senior year; and disaggregation of such 
results by gender, race/ethnicity, and eligibility for free/reduced lunch. Options for state-level 
transcript studies should be considered, from merely sharing the transcript study methodology with 
interested states to federal- state jointly funded special studies. 

NAEP transcript studies also could help answer questions about 12 th grade student motivation on 
NAEP by examining the likelihood that certain subpopulations of students drawn for the NAEP 
sample are more/less likely to be “no-shows” for testing, fail to complete the test booklets, or 
provide “off-task” responses to constructed response questions. 

The Commission members believe that the NAEP transcript studies should be continued and that 
NAEP’s leaders should develop a strategic plan for data collection, analysis, and timely public 
reporting. 



Conclusion 

The Commission members appreciate the opportunity to have reviewed 12 th grade NAEP and have 
come to view it as a precious and unique resource in American education. Because of the 
importance of knowing the achievement of this nation’s 12 th graders, the Commission members 
believe that there is no choice but to change and strengthen 12 th grade NAEP. Continuing to 
provide the public 12 th grade results is essential, expanding state NAEP to 12 th grade will serve a 
crucial need, and providing information about the readiness of 12 th graders for postsecondary 
pursuits will vastly increase NAEP’s usefulness. However, NAEP’s leaders must, with thorough 
resolve and adequate resources, also address the low 12 th grade participation rates and student 
motivation to do their best on NAEP. Only then can NAEP’s ability to “tell the truth” about 12 th 
grade student achievement be fully achieved. 
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Appendix B 



Overview of The National Assessment of Educational Progress 



The National Assessment of Educational Progress (NAEP) is the only nationally representative and 
continuing assessment of what American students know and can do in various academic subjects in 
grades 4, 8, and 12. Authorized and funded by Congress, it is commonly known as the Nation's Report 
Card. Its surveys have been conducted on a national sample basis since 1969 in reading, mathematics, 
science, writing, history, geography, and other elementary and secondary school subjects. NAEP state- 
by-state assessments — also on a representative-sample basis — began in 1990. In 2002 and 2003, 

NAEP conducted the Trial Urban District assessments (TUDA) to study the feasibility of district-level 
reporting as a component of the NAEP program. 

NAEP’s national samples include both public and private schools. The state and TUDA samples are of 
public schools only. Under P.L. 107-110, the No Child Left Behind Act of 2001, all states are required 
to participate in NAEP reading and math assessments in the 4th and 8th grades every two years, 
starting in 2003. Participation in all other NAEP subjects and grades is voluntary. 

The National Assessment Governing Board sets policy and provides general oversight for NAEP. 
The National Center for Education Statistics (NCES) is responsible for administering the project. 

Since 1969, NAEP has produced more than 200 reports in 1 1 instructional areas. By making 
objective information on student performance available to policymakers at the national, state, and 
local levels, NAEP is an important part of our nation's evaluation of the condition and progress of 
education. 

How Is NAEP Carried Out? Through contracts administered by the Governing Board, test 
frameworks and specifications are developed that are the basis for test construction and 
achievement levels are proposed that are the basis for reporting results. Through contracts 
administered by NCES, test instruments are designed based on the test frameworks and 
specifications approved by the Governing Board; samples of students are selected; data are 
collected, scored, and analyzed; and reports are written. 

Who is sampled? NAEP assesses nationally representative samples of public and private school 
students in grades 4, 8, and 12 and state representative samples of public school students in grades 4 
and 8. Scientific sampling procedures are used to ensure that data are reliable and valid. For 
national samples, there are about 8,000 to 10,000 students per subject per grade. For each state 
sample, there are 2,500 to 3,000 students per subject per grade. Schools are randomly selected for 
NAEP based on demographic variables representative of the nation's schools. Students are selected 
randomly. Their confidentiality is protected by law and the names of the students who have 
participated in NAEP do not leave the school. 
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How is NAEP administered? Trained NAEP staff members administer the National Assessment. 
In NAEP state assessments, the participating schools work with a state NAEP coordinator 
designated by the respective state department of education to assist in collecting information on a 
statewide level. 

To minimize the burden on students and schools, no student takes the entire assessment. Instead, the 
200 or more questions developed for each assessment are systematically divided into a set of test 
booklets, with about 40 to 50 questions in each booklet. Each student responds to the questions in a 
single test booklet, which permits student testing time to be limited to 1-1/2 to 2 hours. 
Questionnaires are also given to students, teachers, and principals in order to obtain current 
information about school policies and instructional practices relevant to learning and student 
achievement. Only group statistics are reported and no individual student or teacher data are ever 
released. 

When is NAEP conducted? The NAEP schedule provides for the conduct of assessments each 
year. Most assessments are conducted during the months of January-March. 

What subjects are assessed? The academic subject areas to be assessed are determined by the 
National Assessment Governing Board and set forth to the public in a 10-year schedule. Reading 
and mathematics are assessed once every two years, science and writing once every four years, and 
other subjects (e.g., U.S. history, geography, civics, the arts, foreign language, world history, and 
economics) less frequently. 
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Appendix C 



Commissioned Papers and Research Studies 

1. ACT, Inc; “Analysis of Commonalities and Gaps: Comparing the National Assessment of 
Educational Progress to ACT WorkKeys Assessments.” 

2. Barton, Paul E.; “Grading Twelfth Graders: More Useful and More Used NAEP 
Reporting?” 

3. Carnevale, Anthony P. and Desrochers, Donna M.; “Considerations in Using 12 th Grade 
NAEP as a Prospective Indicator of Readiness for College and Employment.” 

4. DeVito, Pasquale; “Motivation and Incentives for High Schools and High School Students 
to Participate in the National Assessment of Educational Progress at Grade 12. “ 

5. The Education Trust; “What Can Grade 12 NAEP Tell Us about Readiness for College-level 
Work?” 

6. Hoachlander, Gary; “High School Curriculum and Student Outcomes: Building on the 
NAEP High School Transcript Studies.” 

7. Kirst, Michael; “College Preparation and Grade 12 NAEP.” 

8. Mullis, Ina; “Considerations in Framing a Purpose Statement for Who is Tested by the 
National Assessment of Educational Progress at Grade 12.” 



Public Comment 

1. Council of Chief State School Officers; Institute on the High School, Focus Group on 12 th 
Grade NAEP; Driscoll, David P., Smith, Charles E., and Ward, Michael E.; Lake Tahoe, 
Nevada; July 29, 2003. 

2. National Association of State Boards of Education; Annual Conference, Baltimore, 
Maryland; October 16-18, 2003; Presentation and Discussion on Preliminary Proposals for 
Changes to 12 th Grade NAEP; Driscoll, David P., and Musick, Mark D. 

3. National Commission on NAEP 12 th Grade Assessment and Reporting; Presentation and 
Discussion with Tennessee Education Leaders; Nashville, Tennessee; January 14, 2004. 



Presentations Made to the Commission 

Meeting of March 27, 2003: 

1. “What is NAEP?”; Dr. Peggy Carr, Associate Commissioner, National Center for Education 
Statistics. 

2. “New York State Regents Exams”; Dr. Gerald E. DeMauro, New York State Education 
Department. 

3. “High Schools that Work”; Dr. Gene Bottoms, Southern Regional Education Board. 

4. “Oregon PASS”; Dr. David McDonald, Oregon University System. 

5. “Indiana American Diploma Project”; Commissioner Stanley G. Jones, Indiana State Higher 
Education Commission. 

6. “Devolution of the High School Curriculum: 1893 — 1995”; Dr. Jeffrey Mirel, University of 
Michigan. 
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Meeting of March 27, 2003, continued: 

7. “Higher Education Perspective on NAEP”; Dr. David Spence, California State University. 

8. “Military Perspective on NAEP”; Dr. Curtis Gilroy and Dr. Jane Arabian, Department of 
Defense. 

9. “Business Perspective on NAEP”; Dr. Thomas Saterfiel, ACT, Inc. 



Meeting of June 6, 2003: 

1. “What are the NAEP Frameworks and Achievement Levels in Reading, Writing, and 
Mathematics?”; Dr. Mary Crovo and Dr. Sharif Shakrani, National Assessment Governing 
Board. 

2. “What are the NAEP Transcript Studies?”; Dr. Peggy Carr, Associate Commissioner, 
National Center for Education Statistics. 

3. “How Good is ‘Good Enough’ in Reading, Writing, and Mathematics on College Placement 
Tests?”; Ms. Kathy Wanstreet, Southern Il linois University, Edwardsville; Ms. Carolyn 
Kostelecky, ACT, Inc. 



Meeting of September 9, 2003: 

1. “Readiness of Colorado Freshmen for Higher Education”; Timothy E. Foster, Colorado 
Commission on Higher Education. 

2. “Motivation of Students and Incentives for Schools to Participate in 12 th Grade NAEP”; Dr. 
Pasquale DeVito, Assessment and Evaluation Concepts, Inc. 

3. “Strengths and Limitations of NAEP as an Indicator of Readiness for Higher Education”; 
Dr. Michael Kirst, Stanford University. 

4. “Using NAEP Transcript Data of 12 th Graders”; Gary Hoachlander, MPR Associates, Inc. 

5. “Improving Reporting and Use of 12 th Grade NAEP Results”; Paul Barton, consultant. 



Meeting of November 5, 2003: 

1. “Toward a Purpose Statement for 12 th Grade NAEP”; Dr. Ina Mullis, Boston College. 

2. “What Can NAEP Say about Readiness of 12 th Graders for Higher Education?”; Ms. Patte 
Barth, The Education Trust. 

3. “What Can NAEP Say about Readiness of 12 th Graders for Work?”; Dr. Anthony Carnevale, 
ETS; Ms. Jennifer Horn Brown, ACT, Inc. 

4. “What Do We Know about 12 th Grade Achievement, What Do We Need to Know, and What 
Can NAEP Provide?”; Ms. Kati Haycock, The Education Trust. 
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Appendix D 



Increasing Participation In 12 th Grade NAEP: An Example 



Following is one possible scenario, developed by the National Commission on NAEP 12 th Grade 
Assessment and Reporting, to illustrate the factors that would comprise a serious approach to 
increasing the participation of high schools and 12 th grade students in NAEP and the motivation of 
12 th grade students to do their best on NAEP. 



A. Funding 

Incentives for participation will have associated direct and indirect costs. It is essential that these 
costs be acknowledged and fully supported. A specific amount of the annual NAEP appropriation 
should be budgeted and set aside for incentives to schools and students. 



B. Congressional support 

Members of the National Assessment Governing Board, the Secretary of Education, the Director of 
the Institute of Education Sciences, and the Commissioner of Education Statistics, should make a 
compelling case to Congress that 12 th grade NAEP — particularly at the state level — is important, 
that increasing participation is essential, that a comprehensive, effective plan for increasing 
participation has been developed, and that adequate funding for incentives should be supplied. 



C. External support 

Members of the National Assessment Governing Board, the Secretary of Education, the Director of 
the Institute of Education Sciences, and the Commissioner of Education Statistics, should work with 
the following individuals and organizations to explain the importance of 12 th grade NAEP and to 
obtain commitments to assist in increasing participation of high schools and 12 th grade students in 
NAEP: the White House, governors, chief state school officers, state legislators, higher education 
officials, state and national groups representing governors, state legislators, higher education 
officials, chief state school officers, school superintendents, school board members, high school 
principals, teachers, and students, print and broadcast media, entertainment and sports figures, 
advertising and marketing agencies, national and state business groups, and business leaders. 

The staff of the National Center for Education Statistics that administers NAEP should seek advice 
from experts in communications on ways to (1) better “frontload” the process with outreach to state 
superintendents, district superintendents, principals, and teachers; (2) improve the materials that 
explain what NAEP is and why participation is important that the NAEP contractors give to schools 
and students; and (3) improve the “scripts” used with students by test administrators in the testing 
situation. 
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D. Incentives 



Incentives should be developed that are attractive to students. For example, each year, two $5,000 
college scholarships could be awarded per state — one to a female student and one to a male 
student — selected at random from the set of all students who participate that year in 12 th grade 
NAEP, complete all test questions, answer at least 25% of the multiple choice questions in their 
booklet correctly and have no “off task” ratings on constructed response items. Students in fa mi lies 
with incomes that qualify for Pell grants would receive an additional $2,500 scholarship. Upon 
completion of testing, all students who take NAEP would receive a thank you letter from the 
President of the United States on White House stationery. 

In support of this effort, all public 2 and 4 year colleges and universities in each state will offer 20% 
discounts for all bookstore purchases for one year to every student who took 12 th grade NAEP and 
holds the “NAEP discount card.” All 12 th grade students taking NAEP will receive the “NAEP 
discount card,” which will be accepted at college bookstores and also at businesses that support 
NAEP (e.g., national and local bookstores, record stores, clothing stores, retail chains, restaurants, 
movie theaters, etc.). 

An on-going NAEP public information campaign would be conducted. Businesses participating in 
the NAEP discount program will indicate through prominent in-store displays and advertising that 
they support NAEP and encourage students to do their best. As a public service, cooperating 
advertising and marketing agencies, and print and broadcast media will offer discounts to 
businesses that support NAEP and include that message prominently in their advertising and/or in 
public service announcements. 

Partnerships will be sought with entertainment and sports stars — as well as related foundations — 
concerned about education. Public service announcements and personal appearances utilizing these 
figures will be developed and broadcast in areas where NAEP testing will take place, beginning the 
month prior to testing. In addition, a short video using entertainment and sports stars explaining 
what NAEP is and why it is important to do their best will be played for the students as a part of the 
orientation to NAEP prior to testing. 

Incentives for schools will be developed. Incentives that are attractive to students also will be 
attractive to schools, making teachers’ and principals’ “selling” job easier, such as the chance at a 
valuable college scholarship. Letters signed by the President of the United States will be sent to the 
principal of each school selected for the sample, explaining the importance of NAEP at the 12 th 
grade and asking for the principal’s full cooperation, including active, affirmative encouragement of 
students selected for the sample. Participating schools will receive Department of Education 
recognition as a NAEP school with a banner that can be prominently displayed at the school. Local 
print and broadcast media will run stories about the schools that are selected for participation. 
Governors, chief state school officers, and others (e.g., national and state principal groups) will 
fulfill the commitments they made to NAEP’s managers (see above) about encouraging 
participation. For example, they may make direct contacts with the schools in their state to 
encourage participation and also provide letters of support for the sampling and testing contractor to 
include in information packets given to the schools. 
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Summary 

The scenario outlined above attempts to provide one example of a serious, systematic strategy to 
increase school and student participation in grade 12 NAEP. Its key components include: 

• Regular allocation of specified resources 

• Comprehensive identification of key players 

• Multiple, integrated, mutually reinforcing incentives for key players, schools, and 
students 

This example illustrates the resources, imagination, and hard work needed if satisfactory 
improvements in 12 th grade school and student participation in NAEP are to be made. NAEP’s 
leaders must focus seriously on increasing participation or face the prospect that the quality and 
credibility of the 12 th grade NAEP achievement results will be compromised. 
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